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FINAL  REPORT 
Summary  of  Research 

Our  research  supported  by  the  ARO/SDI  grant, 
in  Image  Processing  and  Computer  Vision.  A  general  theme  of  our  work  is  to 
guide  the  development  of  algorithms  and  the  solutions  of  the  computational 
problems  by  carefully  articulated  mathematical  models.  The  modelling  ap¬ 
proaches  have  been  strongly  influenced  by  applications.  The  applications 
stimulate  the  mathematical  research  and,  in  turn,  the  algorithms  derived 
from  the  mathematical  analysis  provide  new  methodological  approaches  for 
the  applications.  Our  main  projects  and  contributions  to  applications  and 
theory,  may  be  divided  into  five  groups: 

1)  Shape  Reconstruction:  We  have  developed  a  coherent  Bayesian 
framework  for  estimating  3D  shapes  (or  topographic  maps),  and  surface 
composition  (albeto,  or  dielectric  properties)  from  a  single  noisy  Video  or 
SAR  image.  Image  1  shows  a  reconstruction  experiment  with  video  data 
acquired  under  uncontrolled  illumination;  Figure  2  shows  a  surface  recon¬ 
struction  from  simulated  SAR  data.  The  method  has  been  applied  to  the 
estimation  of  topographic  maps  of  Venus’  terrain  using  SAR  data  from  the 
Magellan  spacecraft. 

Our  approach  to  the  reconstruction  of  3D  shapes  explores  certain  random 
surface  models,  and  is  formulated  as  a  global  optimization  problem.  To  solve 
this  problem,  we  developed  a  new  multiresolution  algorithm  inspired  from 
recent  studies  in  the  simulation  of  statistical  mechanics  systems  at  criticality. 
The  algorithm  appears  to  be  useful  in  many  image  processing  tasks,  including 
texture  segmentation / classification. 

2)  Texture  Synthesis  and  Classification:  We  have  introduced  a  ap¬ 
pealing  class  of  Markov  Random  Fields  (MRF’s)  models  for  texture  syn¬ 
thesis  and  segmentation/classification.  The  models  are  defined  at  all  levels 
of  resolution  and  are  convenient  for  scale  and  rotation  invariant  segment a- 
tion/discrimination.  Image  3  shows  the  synthesis  of  a  wood-like  texture; 
Image  4  shows  the  segmentation/classification  of  four  textures.  We  have 


two  segmentation  procedures  one  supervised  and  on  unsupervised.  The  mod¬ 
elling  of  textures  via  MRFs  involves  the  estimation  of  certain  parameters 
from  training  data.  We  have  developed  practical  statistical  procedures  for 
parameter  learning,  that  apply  to  a  host  of  other  applications.  The  mathe¬ 
matical  study  of  consistency  and  asymptotic  normality  has  lead  to  a  number 
of  mathematical  problems  and  to  an  interesting  interplay  between  statistical 
inference  and  the  phenomena  of  phase  transitions.  We  have  settled  these 
problems  in  a  number  of  publications. 

3)  Tomography:  We  have  introduced  a  two-stage  EM  algorithm  for  3D 
emission  tomography  to  account  for  blur  due  to  2D  scientilation  cameras; 
the  procedure  makes  the  3D  problem  computationally  feasible.  Also,  we 
developed  a  multi-grid  type  algor1  *hm  for  maximum  likelihood  and  back- 
projections  estimation.  To  our  knowledge,  this  is  the  first  multiresolution 
procedure  in  the  tomography  problem. 

4)  Coding  Theory  for  MRFs  (work  by  Y.  Amit  who  was  partially 
supported  by  this  grant):  the  theory  of  large  deviations  was  used  to  show 
the  number  of  bits  per  symbol  for  Ziv-Lempel  codes  is  given  by  the  maximum 
entropy  of  all  MRFs  with  the  same  potentials.  The  work  provides  also  a  new 
way  for  looking  at  the  classical  1-D  Markov  source  coding  problem. 

5)  Speech  Recognition:  During  this  grant,  we  undertook  a  systematic 
study  of  speech  recognition,  by  exploring  a  new  framework  which  is  con¬ 
ceptually  similar  to,  but  considerably  different  in  details  from  the  Hidden 
Markov  Model  (HMM)  approach.  Our  results  to  date  will  appear  in  a  series 
of  articles  (during  1993/94).  Our  procedure  aims  at  changing  the  acoustic 
model  in  the  HMM  approach  in  a  fundamental  way,  by  using  modern  non- 
parametric/nonlinear  prediction  techniques  to  model  the  acoustic  waveform 
in  the  time  domain.  Our  program  has  lead  to  an  effective  algorithm  for 
discriminating  the  stop  consonants  of  English,  and  to  two  interesting  math¬ 
ematical  problems,  one  of  which  is  a  conceptually  new7  form  of  the  classical 
Wiener-Kolmogorou  prediction  theory. 
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LIST  OF  IMAGES 


Image  1:  (a)  original  image:  an  egg  imaged  under  uncontrolled  illu¬ 
mination  (using  a  desk  lamp),  (b)  reconstruction;  a  matte  surface  was 
assumed;  the  algorithm  estimated  height,  albedo,  and  an  effective  light 
source  direction;  (c)  and  (d)  show  the  reconstructed  scene  illuminated 
from  the  x— ,  y—  direction. 

Image  2:  (a)  plot  of  a  simulated  surface,  (b)  a  32  -  look  SAR  image, 
(c)  plot  of  the  reconstruction  using  (b)  as  the  data. 

Image  3:  Simulation  of  wood-like  texture. 

Image  4:  Segmentation  of  four  textures:  wood,  carpet,  cloth,  plastic 
background. 
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Image  4;  Segmentation  of  four  textures 
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